Highly Scalable Tree-Based Trylock

ABSTRACT

A tree-based trylock technique for reducing contention on a root trylock includes attempting to acquire a trylock at each node of a tree-based hierarchical node structure while following a traversal path that begins at a leaf node, passes through one or more of internal nodes, and ends at a root node having the root trylock. The trylock acquisition operation succeeds if each trylock on the traversal path is acquired, and fails if any trylock on the traversal path cannot be acquired. A trylock housekeeping operation releases all non-root trylocks visited by the trylock acquisition operation, such that if the trylock acquisition operation succeeds, only the root trylock will be remain acquired at the end of the operation, and if the trylock acquisition operation fails, none of the trylocks will be remain acquired at the end of the operation.

BACKGROUND

1. Field

The present disclosure relates to computer systems and methods in whichdata resources are shared among data consumers while preserving dataintegrity and consistency relative to each consumer. More particularly,the disclosure concerns a hierarchical locking technique that may beimplemented as part of a mutual exclusion mechanism known as “read-copyupdate,” or in other computing environments.

2. Description of the Prior Art

By way of background, read-copy update (also known as “RCU”) is a mutualexclusion technique that permits shared data to be accessed for readingwithout the use of locks, writes to shared memory, memory barriers,atomic instructions, or other computationally expensive synchronizationmechanisms, while still permitting the data to be updated (modify,delete, insert, etc.) concurrently. The technique is well suited to bothuniprocessor and multiprocessor computing environments wherein thenumber of read operations (readers) accessing a shared data set is largein comparison to the number of update operations (updaters), and whereinthe overhead cost of employing other mutual exclusion techniques (suchas locks) for each read operation would be high. By way of example, anetwork routing table that is updated at most once every few minutes butsearched many thousands of times per second is a case where read-sidelock acquisition would be quite burdensome.

The read-copy update technique implements data updates in two phases. Inthe first (initial update) phase, the actual data update is carried outin a manner that temporarily preserves two views of the data beingupdated. One view is the old (pre-update) data state that is maintainedfor the benefit of read operations that may have been referencing thedata concurrently with the update. The other view is the new(post-update) data state that is seen by operations that access the datafollowing the update. In the second (deferred update) phase, the olddata state is removed following a “grace period” that is long enough toensure that the first group of read operations will no longer maintainreferences to the pre-update data. The second-phase update operationtypically comprises freeing a stale data element to reclaim its memory.In certain RCU implementations, the second-phase update operation maycomprise something else, such as changing an operational state accordingto the first-phase update.

FIGS. 1A-1D illustrate the use of read-copy update to modify a dataelement B in a group of data elements A, B and C. The data elements A,B, and C are arranged in a singly-linked list that is traversed inacyclic fashion, with each element containing a pointer to a nextelement in the list (or a NULL pointer for the last element) in additionto storing some item of data. A global pointer (not shown) is assumed topoint to data element A, the first member of the list. Persons skilledin the art will appreciate that the data elements A, B and C can beimplemented using any of a variety of conventional programmingconstructs, including but not limited to, data structures defined byC-language “struct” variables. Moreover, the list itself is a type ofdata structure.

It is assumed that the data element list of FIGS. 1A-1D is traversed(without locking) by multiple readers and occasionally updated byupdaters that delete, insert or modify data elements in the list. InFIG. 1A, the data element B is being referenced by a reader r1, as shownby the vertical arrow below the data element. In FIG. 1B, an updater u1wishes to update the linked list by modifying data element B. Instead ofsimply updating this data element without regard to the fact that r1 isreferencing it (which might crash r1), u1 preserves B while generatingan updated version thereof (shown in FIG. 1C as data element B′) andinserting it into the linked list. This is done by u1 acquiring anappropriate lock (to exclude other updaters), allocating new memory forB′, copying the contents of B to B′, modifying B′ as needed, updatingthe pointer from A to B so that it points to B′, and releasing the lock.In current versions of the Linux® kernel, pointer updates performed byupdaters can be implemented using the rcu_assign_pointer( ) primitive.As an alternative to locking during the update operation, othertechniques such as non-blocking synchronization or a designated updatethread could be used to serialize data updates. All subsequent (postupdate) readers that traverse the linked list, such as the reader r2,will see the effect of the update operation by encountering B′ as theydereference B′s pointer. On the other hand, the old reader r1 will beunaffected because the original version of B and its pointer to C areretained. Although r1 will now be reading stale data, there are manycases where this can be tolerated, such as when data elements track thestate of components external to the computer system (e.g., networkconnectivity) and must tolerate old data because of communicationdelays. In current versions of the Linux® kernel, pointer dereferencesperformed by readers can be implemented using the rcu_dereference( )primitive.

At some subsequent time following the update, r1 will have continued itstraversal of the linked list and moved its reference off of B. Inaddition, there will be a time at which no other reader process isentitled to access B. It is at this point, representing an expiration ofthe grace period referred to above, that u1 can free B, as shown in FIG.1D.

FIGS. 2A-2C illustrate the use of read-copy update to delete a dataelement B in a singly-linked list of data elements A, B and C. As shownin FIG. 2A, a reader r1 is assumed be currently referencing B and anupdater u1 wishes to delete B. As shown in FIG. 2B, the updater u1updates the pointer from A to B so that A now points to C. In this way,r1 is not disturbed but a subsequent reader r2 sees the effect of thedeletion. As shown in FIG. 2C, r1 will subsequently move its referenceoff of B, allowing B to be freed following the expiration of a graceperiod.

In the context of the read-copy update mechanism, a grace periodrepresents the point at which all running tasks (e.g., processes,threads or other work) having access to a data element guarded byread-copy update have passed through a “quiescent state” in which theycan no longer maintain references to the data element, assert locksthereon, or make any assumptions about data element state. Byconvention, for operating system kernel code paths, a context switch, anidle loop, and user mode execution all represent quiescent states forany given CPU running non-preemptible code (as can other operations thatwill not be listed here). The reason for this is that a non-preemptiblekernel will always complete a particular operation (e.g., servicing asystem call while running in process context) prior to a context switch.

In FIG. 3, four tasks 0, 1, 2, and 3 running on four separate CPUs areshown to pass periodically through quiescent states (represented by thedouble vertical bars). The grace period (shown by the dotted verticallines) encompasses the time frame in which all four tasks that beganbefore the start of the grace period have passed through one quiescentstate. If the four tasks 0, 1, 2, and 3 were reader tasks traversing thelinked lists of FIGS. 1A-1D or FIGS. 2A-2C, none of these tasks havingreference to the old data element B prior to the grace period couldmaintain a reference thereto following the grace period. All post graceperiod searches conducted by these tasks would bypass B by following theupdated pointers created by the updater.

Grace periods may be synchronous or asynchronous. According to thesynchronous technique, an updater performs the first phase updateoperation, blocks (waits) until a grace period has completed, and thenimplements the second phase update operation, such as by removing staledata. According to the asynchronous technique, an updater performs thefirst phase update operation, specifies the second phase updateoperation as a callback, then resumes other processing with theknowledge that the callback will eventually be processed at the end of agrace period. Advantageously, callbacks requested by one or moreupdaters can be batched (e.g., on callback lists) and processed as agroup at the end of an asynchronous grace period. This allowsasynchronous grace period overhead to be amortized over plural deferredupdate operations.

More recently, RCU grace period processing has been adapted to accountfor processor low power states (such as, on Intel® processors, the C1Ehalt state, or the C2 or deeper halt states). Operating systems can takeadvantage of low power state capabilities by using mechanisms thatwithhold regular timer interrupts from processors (in a low power state)unless the processors need to wake up to perform work. The dynamic tickframework (also called “dyntick” or “nohz”) in existing versions of theLinux® kernel is one such mechanism. In RCU implementations designed forlow power applications in the Linux® kernel, a compiler configurationoption called RCU_FAST_NO_HZ is available. This option allows processorsto be placed in low power states even if there are pending RCUcallbacks, provided none require immediate invocation and the processoris not needed for grace period advancement processing.

One characteristic of the RCU_FAST_NO_HZ option is that quiescent statesare periodically forced in order to expedite callback processing so thatprocessors can enter low power states more quickly. Quiescent stateforcing is regulated by a global lock that serializes access to thequiescent state forcing mechanism. The global quiescent state forcinglock is acquired only with a trylock primitive called“raw_spin_trylock_irqsave( ),” which either immediately acquires thelock or returns failure. As such, the contention on the global quiescentstate forcing lock should be zero. Unfortunately, on large systems(thousands of CPUs), enabling RCU_FAST_NO_HZ has historically beensusceptible to extreme memory contention due to a high rate of attemptsto acquire the global quiescent state forcing lock, resulting inthroughput dropping to nearly zero. This high level of memory contentioncan also result from RCU's implementation of the asynchronous callbackprocessing primitive “call_rcu( ),” which can invoke quiescent stateforcing when large numbers of RCU callbacks are enqueued on the CPU inquestion.

Although an immediate solution is to disable RCU_FAST_NO_HZ on largesystems, this approach results in sub-optimal energy efficiency.Accordingly, there is a need for a technique for acquiring the globalquiescent state forcing lock with reduced memory contention. Other RCUoperations, as well as many non-RCU operations, could likewise benefitfrom a locking technique with reduced lock contention in systems withmany processors.

SUMMARY

A method, system and computer program product are provided to implementtree-based trylock operations that reduce contention on a root trylockin a computer system having two or more processors operatively coupledto one or more memory devices. In an example embodiment, a lockhierarchy is provided in which plural trylocks are distributed amongnodes of a tree-based node structure having a plurality of leaf nodes,one or more internal nodes, and a root node. The processors are assignedto the leaf nodes in a distributed and balanced manner in order tominimize memory contention on the trylocks. A trylock acquisitionoperation is implemented on a selected one of the processors foracquiring a root trylock associated with the root node. The trylockacquisition operation attempts to acquire one of the trylocks at eachnode of the node structure that lies on a traversal path beginning atone of the leaf nodes, passing through one or more of the internalnodes, and ending at the root node. The trylock acquisition operationsucceeds if each trylock on the traversal path is acquired, and fails ifany trylock on the traversal path cannot be acquired. A trylockhousekeeping operation releases all non-root trylocks visited by thetrylock acquisition operation, such that if the trylock acquisitionoperation succeeds, only the root trylock will be remain acquired at theend of the operation, and if the trylock acquisition operation fails,none of the trylocks will be remain acquired at the end of theoperation.

In an example embodiment, the root trylock guards a guarded operationfor which duplicate initiations are redundant, or guards a global lockthat in turn guards the guarded operation.

In an example embodiment, the guarded operation comprises read-copyupdate quiescent state forcing.

In an example embodiment, the trylock acquisition operation furtherincludes checking, at each node on the traversal path, a conditionindicating that an operation protected by the root trylock has alreadybeen initiated by another one of the processors, and failing the trylockacquisition operation if the condition exists.

In an example embodiment, the trylock housekeeping operation isperformed at each of the nodes on the traversal path by releasing atrylock acquired at an immediately preceding node.

In an example embodiment, a global lock acquisition is performed toacquire a global lock after the trylock acquisition operationsuccessfully acquires the root trylock, the root trylock being releasedif the global lock acquisition is successful.

In an example embodiment, the global lock acquisition operation includeschecking a condition indicating that an operation guarded by the globallock has already been initiated by another one of the processors, andfailing the global lock acquisition operation if the condition exists.

BRIEF DESCRIPTION OF THE DRAWINGS

The foregoing and other features and advantages will be apparent fromthe following more particular description of example embodiments, asillustrated in the accompanying Drawings, in which:

FIGS. 1A-1D are diagrammatic representations of a linked list of dataelements undergoing a data element replacement according to aconventional read-copy update mechanism;

FIGS. 2A-2C are diagrammatic representations of a linked list of dataelements undergoing a data element deletion according to a conventionalread-copy update mechanism;

FIG. 3 is a flow diagram illustrating a grace period in which fourprocesses pass through a quiescent state;

FIG. 4 is a functional block diagram showing a multiprocessor computingsystem that may be implemented in accordance with the presentdisclosure;

FIG. 5 is a functional block diagram showing an RCU subsystem that maybe provided in the computer system of FIG. 4;

FIG. 6 is a functional block diagram showing an example RCU nodehierarchy that may be implemented in accordance with the presentdisclosure;

FIG. 7 is a functional block diagram showing components of a leaf RCUnode in the RCU node hierarchy of FIG. 6;

FIG. 8 is a functional block diagram showing components of an internalRCU node in the RCU node hierarchy of FIG. 6;

FIG. 9 is a functional block diagram showing components of a root RCUnode in the RCU node hierarchy of FIG. 6;

FIG. 10 is a functional block diagram showing a set of RCU subsystemsupport functions that be provided by the RCU subsystem of FIG. 5;

FIG. 11 illustrates example C-language code for an RCU force_quiescentstate( ) function that may be implemented in accordance with the presentdisclosure;

FIG. 12 illustrates example C-language code for implementing a hierarchyof tree_lock nodes that each contain a trylock, and for implementing atree_try_lock( ) function, and for implementing a tree_unlock function;

FIG. 13 is a a functional block diagram showing an exampleimplementation of the hierarchy of tree_lock nodes that each contain atrylock, as defined in the example code of FIG. 12;

FIG. 14 is a functional block diagram showing components of a leaftree_lock node in the node hierarchy of FIG. 13;

FIG. 15 is a functional block diagram showing components of an internaltree_lock node in the node hierarchy of FIG. 13;

FIG. 16 is a functional block diagram showing components of a roottree_lock node in the node hierarchy of FIG. 13;

FIG. 17 illustrates example C-language code for using the tree_try_lock() and tree_unlock( ) functions of FIG. 12;

FIG. 18 is a flow diagram showing example operations of thetree_try_lock( ) function of FIG. 12;

FIG. 19 is a flow diagram showing example operations of the code showntree_try_lock( ) function use code shown in FIG. 17; and

FIG. 20 is a diagrammatic illustration showing example computer-readablenon-transitory data storage media that may be used to provide a computerprogram product in accordance with the present disclosure.

DETAILED DESCRIPTION OF EXAMPLE EMBODIMENTS Introduction

The present disclosure provides a hierarchy of conditional trylocks,with a trylock at each node of the hierarchy, including a root trylockat the root node of the hierarchy that guards an operation for whichduplicate initiations are redundant, or which indirectly guards theoperation by guarding a global lock that directly guards the operation.One example of such an operation is quiescent state forcing, asdescribed in the Background section above. In order to reduce contentionon the root trylock, each CPU (or thread) is assigned to one of the leafnodes and conditionally acquires a trylock at each level of thehierarchy in an at-most once manner, beginning at its assigned leaf nodeand moving upwardly through the hierarchy in leaf-to-root fashion.

To conditionally acquire a given trylock, a CPU/thread asserts thetrylock and immediately checks the result. If the acquisition fails, theoverall acquisition fails. Otherwise, if the acquisition succeeds, theCPU/thread moves up to the next level of the hierarchy and repeats.Regardless whether the root trylock acquisition effort succeeds or failsat some point along the node traversal path, all non-root trylocks mustbe released. Releasing these lower level trylocks may be efficientlyhandled by the CPU/thread at each node as it moves upwardly through thelock hierarchy. In particular, after (or before) the CPU/thread attemptsto acquire a trylock at a given level of the hierarchy, it may releasethe prior trylock acquired in the immediately preceding level. Othertrylock release methods may also be used.

If the CPU/thread acquires the trylock at the root of the hierarchy, ithas acquired the root trylock. Additionally, a global lock that protectsa particular operation may be provided as a separate lock that isacquired by the CPU/thread after it acquires the root trylock. In theexample embodiment in which the operation in question is RCU quiescentstate forcing, such an additional global lock is used because it alreadyexists in RCU implementations for the Linux® kernel. The purpose of theglobal lock is to synchronize setting of a flag and a wakeup to thequiescent state forcing operation protected by the lock. In thisembodiment, additional efficiency may be obtained by checking the flagat each level of the hierarchy. If the flag is already set, then aquiescent state forcing operation has already been requested and thereis no need to push further up the hierarchy. Other embodiments maylikewise use this technique, checking the state of a flag (or othercondition indicator) as the lock hierarchy is traversed in order toavoid wasted effort.

EXAMPLE EMBODIMENTS

Turning now to the figures, wherein like reference numerals representlike elements in all of the several views, FIG. 4 illustrates an examplemultiprocessor computer system in which the grace period processingtechnique described herein may be implemented. In FIG. 4, a computersystem 2 includes multiple processors 4 ₁, 4 ₂ . . . 4 _(n), a systembus 6, and a program memory 8. There are also cache memories 10 ₁, 10 ₂. . . 10 _(n) and cache controllers 12 ₁, 12 ₂ . . . 12 _(n)respectively associated with the processors 4 ₁, 4 ₂ . . . 4 _(n). Aconventional memory controller 14 is again associated with the memory 8.As shown, the memory controller 14 may reside separately from processors4 ₂ . . . 4 _(n) (e.g., as part of a chipset).

The computer system 2 may represent any of several different types ofcomputing apparatus. Such computing apparatus may include, but are notlimited to, general purpose computers, special purpose computers,portable computing devices, communication and/or media player devices,set-top devices, embedded systems, and other types of informationhandling machines. The term “processor” as used with reference to theprocessors 4 ₁, 4 ₂ . . . 4 _(n) encompasses any program execution unitcapable of executing program instructions, including but not limited toa packaged integrated circuit device (such as a microprocessor), aprocessing core within a packaged integrated circuit device (such as amicroprocessor core), or a hardware thread comprising one or morefunctional units within a processing core (such as an SMT thread). Eachsuch execution unit may be referred to as a CPU (central processingunit). The processors 4 ₁, 4 ₂ . . . 4 _(n) may be situated within asingle computing device or node (e.g., as part of a single-node SMPsystem) or they may be distributed over plural nodes (e.g., as part of aNUMA system, a cluster, or a cloud). The memory 8 may comprise any typeof tangible storage medium capable of storing data in computer readableform for use in program execution, including but not limited to, any ofvarious types of random access memory (RAM), various flavors ofprogrammable read-only memory (PROM) (such as flash memory), and othertypes of primary storage (i.e., program memory). The cache memories 10₁, 10 ₂ . . . 10 _(n) may be implemented in several levels (e.g., aslevel 1, level 2 and level 3 caches) and the cache controllers 12 ₁, 12₂ . . . 12 _(n) may collectively represent the cache controller logicthat supports each cache level. As illustrated, the memory controller 14may reside separately from processors 4 ₁, 4 ₂ . . . 4 _(n), forexample, as part of a discrete chipset. Alternatively, the memorycontroller 14 could be provided by plural memory controller instancesthat are respectively integrated with the processors 4 ₁, 4 ₂ . . . 4_(n).

Each CPU embodied by a given processor 4 is operable to execute programinstruction logic under the control of a software program stored in thememory 8 (or elsewhere). As part of this program execution logic, updateoperations (updaters) 18 may execute within a process, thread, or otherexecution context (hereinafter “task”) on any of the processors 4. Eachupdater 18 runs periodically to perform updates on a set of shared data16 that may be stored in the shared memory 8 (or elsewhere). In FIG. 4,reference numerals 18 ₁, 18 ₂ . . . 18 _(n) illustrate individual dataupdaters that respectively execute on the several processors 4 ₁, 4 ₂ .. . 4 _(n). As described in the “Background” section above, the updatesperformed by an RCU updater can include modifying elements of a linkedlist, inserting new elements into the list, deleting elements from thelist, and other types of operations. To facilitate such updates, theprocessors 4 may be programmed from instructions stored in the memory 8(or elsewhere) to implement a read-copy update (RCU) subsystem 20 aspart of their processor functions. In FIG. 4, reference numbers 20 ₁, 20₂ . . . 20 _(n) represent individual RCU instances that may periodicallyexecute on the several processors 4 ₁, 4 ₂ . . . 4 _(n). Any givenprocessor 4 may also execute a read operation (reader) 21. Each reader21 runs from program instructions stored in the memory 8 (or elsewhere)in order to periodically perform read operations on the set of shareddata 16 stored in the shared memory 8 (or elsewhere). In FIG. 4,reference numerals 21 ₁, 21 ₂ . . . 21 _(n) illustrate individual readerinstances that may respectively execute on the several processors 4 ₁, 4₂ . . . 4 _(n). Such read operations will typically be performed farmore often than updates, this being one of the premises underlying theuse of read-copy update. Moreover, it is possible for several of thereaders 21 to maintain simultaneous references to one of the shared dataelements 16 while an updater 18 updates the same data element.

During operation of the computer system 2, an updater 18 willoccasionally perform an update to one of the shared data elements 16. Inaccordance the philosophy of RCU, a first-phase update may be performedin a manner that temporarily preserves a pre-update view of the shareddata element for the benefit of readers 21 that may be concurrentlyreferencing the shared data element during the update operation.Following the first-phase update, the updater 18 may register a callbackwith the RCU subsystem 20 for the deferred destruction of the pre-updateview following a grace period (second-phase update). As described in the“Background” section above, this is known as asynchronous grace periodprocessing. Alternatively, the updater 18 may request a synchronousexpedited grace period.

The grace period processing performed by the RCU subsystem 20 entailsstarting new grace periods and detecting the end of old grace periods sothat the RCU subsystem 20 knows when it is safe to free stale data (ortake other actions). Grace period processing may further entail themanagement of callback lists that accumulate callbacks until they areripe for batch processing at the end of a given grace period.

Turning now to FIG. 5, example components of the RCU subsystem 20 areshown. Among these components is a set of RCU subsystem data structures30 that includes an RCU state structure 32 for tracking per-processorquiescent states. For purposes of the present embodiment, it may beassumed that the RCU subsystem 20 is configured as a hierarchical RCUimplementation, as is conventionally provided in large multiprocessorcomputer systems running the Linux® kernel. In that case, the RCUsubsystem data structures 30 will further include a tree 34 of RCU nodesembedded as a linear array within the RCU state structure 32. FIG. 6illustrates an example three-level RCU node tree that includes sixbottom-level leaf RCU nodes 36, three intermediate level internal RCUnodes 38, a top level root RCU node 40.

In addition to the embedded RCU node tree 34, the RCU state structure 32includes a quiescent state indicator 40 for indicating when all of theprocessors 4 have passed through a quiescent state and a grace periodmay be ended. The RCU state structure 32 also includes a quiescent stateforce flag 42 to indicate that a quiescent state forcing operation hasbeen requested by one of the processors 4.

One of the purposes of the RCU node tree 34 is to reduce contention forthe quiescent state indicator 40, which could be problematic if theindicator was protected by a single global lock. See P. McKenney,Hierarchical RCU, <lwn.net/Articles/305782>, Nov. 4, 2008. Instead ofall the processors 4 using a global lock to access the quiescent stateindicator 40, subsets of the processors are assigned to individual leafRCU nodes 36. Similarly, subsets of the leaf RCU nodes 36 are assignedto individual internal RCU nodes 38. The internal RCU nodes 38 arelikewise assigned to the root RCU node 40. As shown in FIG. 7, each leafRCU node 36 maintains an array 36A of quiescent state bits, with eachbit corresponding to one of the leaf node's assigned processors. By wayof example, FIG. 7 shows four quiescent state bits associated with fourof the processors 4. A bit array lock 36B serializes access to thequiescent state bit array 36A. Similarly, each internal RCU node 38maintains an array 38A of quiescent state bits, with each bitcorresponding to one of the internal node's assigned RCU leaf nodes 36.A bit array lock 38B serializes access to the quiescent state bit array38A. The root RCU node 40 maintains an array 40A of quiescent state bitsto which the internal RCU nodes 38 are respectively assigned. A bitarray lock 40B serializes access to the quiescent state bit array 40A.

When a processor 4 passes through a quiescent state, it asserts the RCUnode lock 36B in its assigned leaf RCU node 36, and sets its assignedquiescent state bit in the bit array 36A. To propagate the quiescentstate information upwardly through the RCU node tree 34, the lastprocessor 4 to set its bit in a leaf RCU node 36 acquires the RCU nodelock 38B in the internal RCU node 38 to which the leaf node is assigned,and sets the internal node's quiescent state bit in the bit array 38A.In similar fashion, the last processor 4 to set a bit in an internal RCUnode 38 acquires the RCU node lock 40B in the root RCU node 40, and setsthe root node's quiescent state bit in the bit array 40A. Finally, thelast processor 4 to set a bit in the root node bit array 40A accessesand sets the quiescent state indicator 42, thereby signaling that thecurrent RCU grace period may end.

It should be noted that a production read-copy update implementationwill typically include many additional data structures that are notshown in FIG. 5. A discussion of such data structures is omitted forease of description and in order to focus attention on the tree-basedtrylock technique disclosed herein. As will be described in more detailbelow, the RCU node tree 34 may be conveniently used to provide ahierarchical tree structure for the new trylock technique. Otherhierarchical trees may be used in other implementations.

Returning now to FIG. 5, the components of the RCU subsystem 20 alsoinclude several RCU subsystem support functions 50, namely, an RCUreader API (Application Programming Interface) 52, an RCU updater API54, and a set of grace period detection and callback functions 56.

As shown in FIG. 10, the RCU reader API 52 comprises a readerregistration component 52A and a reader unregistration component 52B.These components are respectively invoked by readers 21 as they enterand leave their RCU read-side critical sections. This allows the RCUsubsystem 20 to track reader operations and determine when readers areengaged in RCU-protected read-side critical section processing. In anexample embodiment, the reader registration component 52A and the readerunregistration component 52B may be respectively implemented using thercu_read_lock( ) and rcu_read_unlock( ) primitives found in existingread-copy update implementations.

As also shown in FIG. 10, the RCU updater API 54 comprises a registercallback component 54A. The register callback component 54A is used byupdaters 18 to register a callback following a first-phase update to ashared data element 16. An invocation of the register callback component54A initiates processing that places the callback on one of the RCUcallback lists 34 associated with the processor 4 that runs the updater18. This may start an asynchronous grace period (if one is not alreadyunderway) so that the callback can be processed after the grace periodhas ended as part of second-phase update processing to remove stale data(or perform other actions). In an example embodiment, the registercallback component 54A may be implemented using the existing call_rcu( )primitive found in conventional read-copy update implementations.

With continuing reference to FIG. 10, the grace period detection andcallback processing component 56 of the RCU subsystem 20 include a forcequiescent state component 58 that implements the tree-based trylocktechnique disclosed herein. The grace period detection and callbackprocessing component 56 also includes a number of other conventional RCUcomponents 64 that are responsible for various operations, such asstarting new grace periods, detecting the end of old grace periods, andprocessing callbacks as grace periods end. A discussion of suchcomponents is omitted for ease of description and in order to focusattention on the tree-based trylock technique disclosed herein.

As described in the Introduction section above, the tree-based trylocktechnique disclosed herein contemplates a hierarchical tree of nodes,with a trylock at each node, and including a root trylock at the rootnode of the hierarchy. As is known in the art, a trylock is a lockhaving at-most-once semantics, such that lock acquisition is tried onlyonce and either succeeds or fails. The processors 4 are assigned todifferent leaf nodes in a distributed and balanced manner in order tominimize memory contention. A CPU or a thread running within a CPU thatdesires to acquire the root trylock begins at one of the leaf nodes ofthe lock hierarchy, and works its way to the root trylock node followinga traversal path, conditionally acquiring trylocks at lower level nodesas it does so, and releasing all trylocks and abandoning the acquisitioneffort if it fails to acquire a trylock at any given level. As statedabove, in a hierarchical RCU implementation as described above inconnection FIGS. 5-9, the RCU nodes 36, 38 and 40 of the RCU nodehierarchy 34 provide convenient data structures for holding thetree-based trylocks contemplated by the present disclosure. Suchtree-based trylocks are shown in the RCU node diagrams of FIGS. 7, 8 and9, and are identified as force quiescent state (FQS) locks insofar asthe present embodiment uses the trylocks to serialize access to thequiescent state force flag 44 shown in FIG. 5. In FIG. 7, the leaf RCUnode 36 is shown as having a force quiescent state trylock 66. In FIG.8, the internal RCU node 38 is shown as having a force quiescent statetrylock 68. In FIG. 9, the root RCU node 40 is shown as having a rootforce quiescent state trylock 70. In addition, the root RCU node 40includes a global force quiescent state 72 (which may or may not be atrylock), that protects access to the quiescent state force indicator44.

With reference now to FIG. 11, example C-language pseudocode that may beused to implement the force quiescent state component 58 is shown. Thefunction name given to the quiescent state component 58 is“force_quiescent state( ).” The argument to this function is a pointer“*rsp” to the RCU state structure 32 shown in FIG. 5. The purpose of thefunction is to set the quiescent state force flag 44, which is given thename “GP_FLAGS_FQS” in the code, and is implemented as a bit in flagsfield of the RCU state structure 32 called “rsp->gp_flags.” Line 8 ofthe code disables interrupts in order to pin the current task onto aparticular CPU, allowing line 9 to obtain a stable reference to thisCPU's leaf RCU node 36 (named “rnp”). As an alternative, it is possibleto take a snapshot of the CPU number without disabling interrupts. Thisalternative approach risks additional cache misses, but provides betterscheduling latency. Moreover, a user-level implementation of thedisclosed tree-based trylock technique would normally be incapable ofdisabling interrupts.

The loop spanning lines 10-21 traverses the RCU node tree 34 (see FIG.6) from leaf to root. Line 11 checks the quiescent state flag 44 to seeif another task already performed the required work, and if not, line 12attempts to acquire the current RCU node's force quiescent state trylock(see elements 66, 68 and 70 of FIGS. 7, 8 and 9, respectively), referredto in the code as “fqslock.” To expedite the memory access, each forcequiescent state trylock 66, 68 or 70 may be aligned to its own cacheline. The Boolean result of lines 11 and 12 is stored in the “ret”variable. If line 13 determines that a trylock acquired during theprevious pass through the loop is still being held, line 14 releases itusing the tree unlock component 62 of FIG. 10. Line 15 tests the “ret”variable. If the line 11 determined that the quiescent state flag 44 isalready set, or if the attempt on line 12 to acquire the force quiescentstate trylock 66, 68 or 70 failed, line 16 increments a statisticalcounter (which can lose counts), line 17 re-enables interrupts, and line18 returns to the caller. Otherwise, line 20 prepares for the next passthrough the loop.

Upon exit from the loop, the root RCU node 40 will have been reached andthe root force quiescent state trylock 70 will have been acquired. Line22 unconditionally acquires the RCU node's global force quiescent statelock 72, whose name in the code is simply “lock.” Line 23 releases theroot force quiescent state lock 70 using the tree unlock component 62 ofFIG. 10. Line 24 optionally makes one final check of the quiescent stateforce flag 44 to see if it is already set by another task, and if so,lines 25-27 increment the statistical counter, release the root RCUnode's global force quiescent state lock 72, and return to the caller.Otherwise, line 29 sets the quiescent state force flag 44, line 30releases the root RCU node's global force quiescent state lock 72, andline 31 wakes up a thread that attempts to force quiescent states inorder to end the current grace period.

It is possible to abstract the forgoing tree-based trylock operationsfor use in other environments, it being understood that the RCU forcequiescent state scenario described above is merely on example embodimentof the disclosed subject matter. More generally, any case where a largenumber of CPUs might need to awaken or otherwise signal a specificCPU/thread to perform an operation with at-most-once semantics can usethe disclosed mechanism. This sort of arrangement offers cache-localityadvantages (and also a reduction in synchronization overhead) in caseswhere a specific type of processing can be guaranteed to never consumemore than one CPU's worth of processing.

FIGS. 12-16 illustrate one example abstraction of the tree-based trylockoperations performed by the force_quiescent_state( )function of FIG. 11(which in turn is an example implementation of the force quiescent statecomponent 58 of FIG. 10).

The first few lines of FIG. 12 illustrates an abstract “tree_lock” nodethat is analogous to the RCU nodes 36, 38 and 40 of the RCU node tree 34of FIG. 6. FIG. 12 also illustrates an abstract “tree₁₃ try_lock( )”function that is analogous to the loop spanning lines 9-21 of theforce_quiescent state( ) function in FIG. 11. As previously described,this loop attempts to acquire the root RCU node's force quiescent statetrylock 70 shown in FIG. 9. FIG. 12 likewise illustrates an abstract“tree_unlock( )” function that is analogous to the “raw_spin_unlock( )”function on line 23 in FIG. 11 that unlocks the root RCU node's forcequiescent state trylock 70.

Lines 1-6 of FIG. 12 show the structure of the aforementioned“tree_lock” node. A set of tree_lock nodes is arranged hierarchically ina tree of such nodes (a “tree_lock tree”), one example of which is shownby reference number 80 in FIG. 13. The tree_lock tree 80 includes a setof leaf tree_lock nodes 82, a set of internal tree_lock nodes 84 and atop level root tree_lock node 86. As shown in line 2 of FIG. 12, each ofthe tree_lock nodes 82, 84 and 86 contains a trylock that is named“try_lock.” As shown in FIG. 14, the trylock in the leaf tree_lock nodes82 is identified by reference number 88, the trylock in the internaltree_lock nodes 84 is identified by reference number 90, and the trylockin the root tree_lock node 82 is identified by reference number 92 (theroot trylock). As shown in line 3, each of the tree_lock nodes 82, 84and 86 also contains a reference to the node's immediate parent, withthe root node's ->parent field being NULL. As shown in line 4, each ofthe lower level tree_lock nodes 82 and 84 also contains a reference tothe root tree_lock node 86, as does the root tree_lock node itself. Asshown in line 5, each of the tree_lock nodes 82, 84 and 86, mayoptionally contain a “failcount” field. This field is used as a failurecounter that is analogous to the statistics counter used in lines 16 and25 of FIG. 11, except that it is maintained on a per-node basis ratherthan globally as in FIG. 11. The procedures to initialize the fulltree_lock tree 80 and to point threads at their respective leaf nodesare straightforward tasks that will not be belabored here.

The tree_try lock( ) function shown on lines 8-26 of FIG. 12 traversesthe tree_lock tree 80 in an effort to acquire the root trylock 92. The“*check( )” function pointer passed as an argument in line 9 isanalogous to the check in line 11 of the force_quiescent_state( )function of FIG. 11 to determine if a flag or other variable guarded bythe root lock being acquired is already set. The check( ) function takes*arg as its sole parameter. The loop spanning lines 15-24 traverses thetree_lock tree 80 from leaf to root, starting with a specified leaftree_lock node 82 passed as an argument to the tree_try_lock( )function. Line 16 sets local variable “ret” to true if either the check() function determines that the required work is already done or the“spin_trylock( )” function fails to acquire the current tree_lock node'strylock (88, 90 or 92), either of which will terminate the traversal upthe tree_lock tree 80. Lines 17 and 18 release the trylock acquired (ifany) during the previous pass through the loop. Line 20 increments thisnode's failure counter and line 21 returns to the caller to indicatefailure. Otherwise, line 23 prepares for the next pass through the loop.If these operations succeed in locking the root trylock 92, line 25reports success to the caller. Lines 28-33 show the tree_unlock( )function, which must be called after the tree_try_lock( ) functionreports success. This function simply releases the root trylock 92.

An example usage of the tree-based try lock technique of FIGS. 12-16 isshown in FIG. 17. Line 2 of FIG. 17 declares the variable protected bythe root trylock 92 (a flag variable called “myflag” that is analogousto the gp_flags variable checked in lines 11, 24 and 29 of FIG. 11).Line 1 defines the value to which myflag is be set in order to initiatesome operation for which duplicate initiations are redundant. This valueis defined as a flag called “MY_BIT” that has magnitude of 0×1. TheMY_BIT value is analogous to the RCU_GP_FLAG_FQS value checked in lines11, 24 and 29 of FIG. 11. Line 3 of FIG. 17 declares a global lockcalled “my_lock” that protects the myflag variable. This global lockcould be located anywhere in memory, including in the root tree_locknode 86 if desired. Optionally, the global lock could be eliminated andthe root trylock 92 could be used exclusively to guard the myflagvariable.

FIG. 17 also illustrates an abstract “my_check( )” function and a“set_my_bit( )” function. The my_check( ) function is defined in lines7-12. This is the check( ) function that is passed as an argument totree_try_lock( ) function, as discussed above in connection with FIG.12. Its purpose is to check if the variable protected by the globalmylock (in this case the myflag variable) has already been set to thedesired value (in this case the MY_BIT flag). The set_my_bit( ) functionis defined in lines 14-26. This function uses the tree_try_lock( ) andtree_unlock( ) functions of FIG. 12 to set the MY_BIT flag in the myflagvariable. Line 16 invokes tree_try_lock( ) and line 17 returns if itfails (indicating that the root trylock 92 could not be acquired, orthat the required work was already done by another task). Line 18acquires the global mylock that protects the myflag variable, and line19 invokes tree_unlock( ) to release the root trylock 92. Note that thecall to tree_unlock( ) may be deferred to follow line 25 if reducingmemory contention on the global mylock is more important thanacquisition latency on this lock. In fact, the call to tree_unlock( )could be placed anywhere between its current position and line 26. Lines20-23 do one last check to see if the MY_BIT flag has already been setin the myflag variable. Note that these lines may be omitted if memorycontention on the myflag variable is not a concern. Line 24 sets MY_BIT,and could be optionally followed by code that awakens some other threador process to perform a desired operation. Finally, line 25 releases theglobal mylock.

The code of FIGS. 12-17 shows how the disclosed tree-based trylocktechnique may be used to efficiently and scalably provide the“do-at-most-once” semantics that are required for posting new work to athread that implements an operation for which duplicate initiations areredundant.

Turning now to FIGS. 18 and 19, flow diagrams are shown to furtherelucidate operations of the tree_try_lock( ) function and the set_my_bitfunction( ). Operations that are considered optional, such as the check() function and statistics logging, are shown in dashed-linerepresentation. It is assumed that a tree_lock tree, such as that shownin FIG. 13, has been established and initialized. Block 100 of FIG. 18(illustrating operations of the tree_try_lock( ) function) starts theloop of lines 15-26 in FIG. 12. So long as the root tree_lock node 86has not been processed, processing proceeds to block 102, which releasesthe trylock of the previous tree_lock node so long as the current nodeis not the initial leaf tree_lock node 82 (see line 18 of FIG. 12).Preferably, the lower level trylock is not released until after the nextblock 104 is implemented. However, the lower level trylock release canbe performed beforehand. Blocks 104 and 106 implement line 16 of FIG.12. Block 104 attempts to acquire the trylock belonging to the currenttree_lock node. Optional block 106 checks to see if the MY_BIT flag hasalready been set. If the trylock acquisition is unsuccessful or if theMY_BIT flag is set, optional block 108 increments the current tree_locknode's failcount counter (see line 20 of FIG. 12) and block 110 returnsfailure (see line 21 of FIG. 12). If the trylock acquisition in block104 is successful, and if the MY_BIT flag is not already set, block 112prepares for the next pass through the loop (see line 23 of FIG. 12).Assuming the loop successfully processes the root tree_lock node 86 inthe foregoing manner, block 114 returns success to the caller (see line25 in FIG. 12).

Turning now to FIG. 19, which illustrates the set_my_bit( ) function ofFIG. 17, block 120 attempts to invoke the tree_try_lock( ) function, andreturns failure in block 122 if unsuccessful (see lines 16-17 of FIG.17). If the root trylock 92 is acquired, block 124 acquires the globalmylock guarding the variable of interest, namely, the myflag variable(see line 18 of FIG. 17). Note that as an alternative to using theglobal mylock, the root trylock 92 could serve as the global lock thatprotest the myflag variable. Block 126 releases the root trylock 92 (seeline 19 of FIG. 17). Optional block 128 checks to see if the MY_BIT flagis already set, and if it is, block 130 releases the global mylock andblock 132 returns to the caller (see lines 20-22 in FIG. 17). Assumingthe MY_BIT flag is not set, block 134 sets the MY_BIT flag in the myflagvariable (see line 24 of FIG. 17), block 130 releases the global mylock,and block 132 returns to the caller (see line 25 of FIG. 17).

Accordingly, a highly scalable tree-based trylock technique has beendisclosed. It will be appreciated that the foregoing concepts may bevariously embodied in any of a data processing system, a machineimplemented method, and a computer program product in which programminglogic is provided by one or more machine-readable non-transitory datastorage media for use in controlling a data processing system to performthe required functions. Example embodiments of a data processing systemand machine implemented method were previously described in connectionwith FIG. 4-19. With respect to a computer program product, digitallyencoded program instructions may be stored on one or morecomputer-readable non-transitory data storage media for use incontrolling a computer or other digital machine or device to perform therequired functions. The program instructions may be embodied as machinelanguage code that is ready for loading and execution by the machineapparatus, or the program instructions may comprise a higher levellanguage that can be assembled, compiled or interpreted into machinelanguage. Example languages include, but are not limited to C, C++,assembly, to name but a few. When implemented on a machine comprising aprocessor, the program instructions combine with the processor toprovide a particular machine that operates analogously to specific logiccircuits, which themselves could be used to implement the disclosedsubject matter.

Example computer-readable non-transitory data storage media for storingsuch program instructions are shown by reference numerals 8 (memory) and10 (cache) of the computer system 2 of FIG. 4. The system 2 may furtherinclude one or more secondary (or tertiary) storage devices (not shown)that could store the program instructions between system reboots. Afurther example of computer-readable non-transitory data storage mediathat may be used to store the program instructions is shown by referencenumeral 200 in FIG. 20. The data storage media 200 are illustrated asbeing portable optical storage disks of the type that are conventionallyused for commercial software sales, such as compact disk-read onlymemory (CD-ROM) disks, compact disk-read/write (CD-R/W) disks, anddigital versatile disks (DVDs). Such data storage media can store theprogram instructions either alone or in conjunction with an operatingsystem or other software product that incorporates the requiredfunctionality. The computer-readable non-transitory data storage mediacould also be provided by other portable data storage media (such asfloppy disks, flash memory sticks, etc.), or data storage media combinedwith drive systems (e.g. disk drives). As is the case with the memory 8and the cache 10 of FIG. 4, the computer-readable non-transitory datastorage media may be incorporated in data processing platforms that haveintegrated random access memory (RAM), read-only memory (ROM) or othersemiconductor or solid state memory, all of which represent furtherexamples of computer-readable non-transitory data storage media. Morebroadly, the computer-readable non-transitory data storage media couldcomprise any electronic, magnetic, optical, infrared, semiconductorsystem or apparatus or device, or any other tangible entity representinga machine, manufacture or composition of matter that can contain, store,communicate, or transport the program instructions for use by or inconnection with an instruction execution system, apparatus or device,such as a computer. For all of the above forms of computer-readablenon-transitory data storage media, when the program instructions areloaded into and executed by an instruction execution system, apparatusor device, the resultant programmed system, apparatus or device becomesa particular machine for practicing embodiments of the method(s) andsystem(s) described herein.

Although various example embodiments have been shown and described, itshould be apparent that many variations and alternative embodimentscould be implemented in accordance with the disclosure. It isunderstood, therefore, that the invention is not to be in any waylimited except in accordance with the spirit of the appended claims andtheir equivalents.

1-7. (canceled)
 8. A multiprocessor system, comprising: two or moreprocessors; a memory coupled to said processors, said memory including acomputer useable medium tangibly embodying at least one program ofinstructions executable by said processors to implement a read-copyupdate (RCU) subsystem and to perform tree-based trylock operations thatreduce contention on a root trylock, said operations comprising:providing a lock hierarchy which plural trylocks are distributed amongnodes of a tree-based node structure having a plurality of leaf nodes,one or more internal nodes, and a root node; assigning said processorsto said leaf nodes in a distributed and balanced manner in order tominimize memory contention on said trylocks; implementing a trylockacquisition operation on a selected one of said processors for acquiringa root trylock associated with said root node; said trylock acquisitionoperation including attempting to acquire one of said trylocks at eachnode of said node structure that lies on a traversal path beginning atone of said leaf nodes, passing through one or more of said internalnodes, and ending at said root node; said trylock acquisition operationsucceeding if each trylock on said traversal path is acquired, andfailing if any trylock on said traversal path cannot be acquired; andperforming a trylock housekeeping operation that releases all non-roottrylocks visited by said trylock acquisition operation, such that ifsaid trylock acquisition operation succeeds, only said root trylock willbe remain acquired at the end of said operation, and if said trylockacquisition operation fails, none of said trylocks will be remainacquired at the end of said operation.
 9. The system of claim 8, whereinsaid root trylock guards a guarded operation for which duplicateinitiations are redundant, or guards a global lock that in turn guardssaid guarded operation.
 10. The system of claim 9, wherein said guardedoperation comprises read-copy update quiescent state forcing.
 11. Thesystem of claim 8, wherein said trylock acquisition operation furtherincludes checking, at each node on said traversal path, a conditionindicating that an operation protected by said root trylock has alreadybeen initiated by another one of said processors, and failing saidtrylock acquisition operation if said condition exists.
 12. The systemof claim 8, wherein said trylock housekeeping operation is performed ateach of said nodes on said traversal path by releasing a trylockacquired at an immediately preceding node.
 13. The system of claim 8,wherein said operations further include performing a global lockacquisition to acquire a global lock after said trylock acquisitionoperation successfully acquires said root trylock, said root trylockbeing released if said global lock acquisition is successful.
 14. Thesystem of claim 1, wherein said global lock acquisition operationincludes checking a condition indicating that an operation guarded bysaid global lock has already been initiated by another one of saidprocessors, and failing said global lock acquisition operation if saidcondition exists.
 15. A computer program product, comprising: one ormore machine-readable non-transitory data storage media; programinstructions provided by said one or more data storage media forprogramming a multiprocessor data processing platform to implement aread-copy update (RCU) subsystem and to perform tree-based trylockoperations that reduce contention on a root trylock, said operationscomprising: providing a lock hierarchy which plural trylocks aredistributed among nodes of a tree-based node structure having aplurality of leaf nodes, one or more internal nodes, and a root node;assigning said processors to said leaf nodes in a distributed andbalanced manner in order to minimize memory contention on said trylocks;implementing a trylock acquisition operation on a selected one of saidprocessors for acquiring a root trylock associated with said root node;said trylock acquisition operation including attempting to acquire oneof said trylocks at each node of said node structure that lies on atraversal path beginning at one of said leaf nodes, passing through oneor more of said internal nodes, and ending at said root node; saidtrylock acquisition operation succeeding if each trylock on saidtraversal path is acquired, and failing if any trylock on said traversalpath cannot be acquired; and performing a trylock housekeeping operationthat releases all non-root trylocks visited by said trylock acquisitionoperation, such that if said trylock acquisition operation succeeds,only said root trylock will be remain acquired at the end of saidoperation, and if said trylock acquisition operation fails, none of saidtrylocks will be remain acquired at the end of said operation.
 16. Thecomputer program product of claim 15, wherein said root trylock guards aguarded operation for which duplicate initiations are redundant, orguards a global lock that in turn guards said guarded operation.
 17. Thecomputer program product of claim 16, wherein said guarded operationcomprises read-copy update quiescent state forcing.
 18. The computerprogram product of claim 15, wherein said trylock acquisition operationfurther includes checking, at each node on said traversal path, acondition indicating that an operation protected by said root trylockhas already been initiated by another one of said processors, andfailing said trylock acquisition operation if said condition exists. 19.The computer program product of claim 15, wherein said trylockhousekeeping operation is performed at each of said nodes on saidtraversal path by releasing a trylock acquired at an immediatelypreceding node.
 20. The computer program product of claim 15, whereinsaid operations further include performing a global lock acquisition toacquire a global lock after said trylock acquisition operationsuccessfully acquires said root trylock, said root trylock beingreleased if said global lock acquisition is successful.